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ABSTRACT 

A detailed explicit definition of learning disability 
(LD) is presented in terms of the diagnostic procedures and decision 
points' used in identifying that category amdng^a group of jiivenile 
delinquent and control group adolescent males. The described 
procedures include a review of existing records, interviews, and 
diagnostic assessments. Rationales for the choice of pertinent • 
variables and measurement instruments are given. The assessment 
process and administrative arrangements are explained. The factor 
analytic studies of HitJcin and his colleagues were deemed especially 
relevant and the Hechsler Intelligence Scale for Children (Revised) 
vas used. Decision points eliminated ^oys who were mentally retarded 
(intelligence quotient below 75) , and boysiwho were, or had been, 
diagnosed as emotionally disturbed (those diagnosed as untestable or 
presenting bizarre symptoms) . Those chosen for further diagnosis 
included boys whose achievement score differed by two or more years 
from their ability score, or boys whose academic achievement was two 
or more years retarded. Interviews and diagnostic testing were 
carefully standardi;Sed and required about three hours for each boy. 
Training, quality control, and other administrative procedures are 
described. (CTM) 
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Foreword 




It 18: hoped that .this documentation of the procedures that were followed will be 
useful to" researchers, clinicians and others who are concerned with making 
deterMnatloos regarding the presence of learning disabilities (LD) In adolescent 

The two-year program was funded In October, 1976, by the National Institute 
for Juvenile Justice and Delinquency Prevention of the Law Enforcement Assistance 
Administration through grants to the Assoclatldn for Children with Learning - 
Disabilities (ACLD) and Crelgh^ton University's Institute for Business, Law and 
Social Research. Creighton University contracted with Educational Testing 
Service to perform the diagnostic evaluations of the participants in the study. 
The program has three major components: (1) the determination of the prevalence 
of LD in groups of adjudicated delinquents and offlclaily nondellnquent 12-to-15 
year old boys; (2) a remediation {Program for selected groups of adjudicated 
delinquents who are judged LD; (3) and an evaluation of the effectiveness of the 
remediation program. /• 

The urogram is being conducted primarily in Phoenix, Arizona; Indianapolis, 
Indiana; and Baltimore, Maryland. The lnclden<;e study and the evaluation are 
being conducted by the Creighton Institute with Educational Testing Service, 
while the remediation program is the respoosibility of the ACLD. 

The incidence study will provide data systematically concerning the preva^ 
lence of LD in adolescent males, and will be used to select participants ^for the 
remediation program. An opjerational , definition of LD has been adopted for the 
study, and procedures and criteria for making diagnostic judgments have been 
specif led. The explication of the definition and the decision process is the 
focus of this paper. Preliminary results of the Incidence study will be released 
early in 1978. , 

We believe that this effort is unique in the degree to which the^ criteria 
for making diagnostic decisions have been specified explicitly* We realize that 
no single document can be the "final word" in any field as dynamic as the study 
of learning disabilities; however, we hope that this documentation will be useful 
in future efforts* 




Paul K. Broder 
Creighton University 
July, 1977 
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I. OVERVIEW, * 

The task addressed by these procedures can best be considered in, several 

> ' ., , * • ' 

parts. In sequence, they Include reviewing records and applying decision rules. 

for the selection of students for further diagnostic assessments, conducting 
diagnostic asses anient, making recommendations for remediation, initial analysis 
and post-remcdlatlon data collection. Paralleling t-hese atojot procedures are 
several actlylties^ including the briefing of assessors, »quality control, and 
information reporting to several audiences. 

There were certain constraints which were operating during the Implementation 
of 'the study .which mtfbt be^'kept in mind when one considers tfie selected proce^ 
dures. Two of them have h Ad significant, and to some extent, controlling influ- 
ence on the procedures design.. First is the nebulousness of and lack *of agreement 
on current definitions of learning disability. The t)ropo8ed rules (Federal 
Register, 1976) governing th^ allocation of ^deral assistance to states, for 
example. Include the following comments: 

1. The state of che art in the field of specific learning ^ 
disabilities and its associated fields Is such that it is not 

- presently possible' to specify exactly all of the components 
of each specific learning disability. There remain strong 
opposing professional opinions as to the validity of a 
specific learning disability. At present, the only generally ^ 
i accepted manifestation of a specific learning disability is 
that there is a major discrepancy between expected achieve- 
ment and ability which is not the result of other known and - 
generally accepted h>andlcapplng conditions or circumstances. 

2. There exists no hard research data collected on a large 
enough sample in order to state, with certainty, which are. 
the common characteristics of all learning disabled children. 

/ 7 

3. . There -are several theories as/to what causes children to have 

specific learning disabilities. 

} ■ ' ' 

4. There appear to be no generally accepted diagnostic instru- 
ments pMj^ntly available which ca:n be singly and approprl- 

.ately utl^Pxed with all children with a specific learning 
disability. 
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This analysis of the definitional problem is aldo supported by the findings 
of the GM^ral Accounting Office (Fogel, R.L., et al , 1976). They reported that 
studies reviewed showed very wi^e variations of definitions and, as J result^ th^y 
developed an operational definition of the ability-achievement discrepancy 
type. 

Ideally, under these circumstances one should collect, from a represent'&t ive 
sample, data descriptive of several plausible definitions; apply construct 
validity analyses to these data, from these rpsults determine the definitionfi 
best supported amoirlcaliy, and then collect fa new set of data from a new sample 



to determine incl3Sbnce rates and provide remedial reco^i^ndation. The time and 
resources available for this study did not permit such an approach. Therefore, an 
a priori definition,, based upon profile discrepancy in learning and ability, has 
been used to select d measurement package, and thereby to determine the incidence o 

learning disability in terms of such a definition. Subsequent analysis bt the 

c 

collected data is expected to provide the opportunity to refine the definition 
empirically, thus adding to the accumulated knowledge about learning disability, 
and if resources permit, making it possible to reanalyze the cdliected protocols/ 
for revision of the incidence estimates. 

9 

The second constraint was imposed by the impracticality of brd^ngirig the 
students who participated in the study to the testing site oi^ more than one 
occasion. Individual testing consumes a great deal of professional time, thereby 
consuming a major portion of the study's resources. It' is also common knowledge 
that a period of two and one-half to three hours is the maximum for a session 
with, a single individual. As a result of this situation, the study was limited to 
the data available from each student's records and from an assessment session 
completed in a two-and-one-half- to three-hour period. Within these two constraint! 

^ 



the proc^durea vere designed to produce the maxlmum-of usable Information and to. 
review -this information in a sye^ematic. and replicabiy^wnner.' Thft nature of the 
problem required the exercise of -c^onsiderabi^ prof essicJnal judgment , within ' • 
guidelines J[hich could be applied in a variety of settings. 

The details of the procedures a're presented in the following pages. 



8 



II. THEORETICAL APPROAOV^TO THE ASSESSMENT OF LEARNIilG DISABILITIES 

■'• ' ' ' * ' / ; 

th« procedure 4**crlb«d ^ere represents an approach necessitated by scu^ 
prl^o^lties. An^early start for th^e treatment phase of the project was imperatlvee 
The apprbach of providing a' priori definition and decision rules for diagrioais 
s^eofl;d the only way to provide potedtial treatment of subjection schedule. The 
data* considered must, of. course, b^e^'tightly focusej on the a ptlorl definition 
rather tj^an brdadly £oclused I'd inore general domains. Second, because individual 
diagnostic decisions had to be made as soon as possible following data collection, 
it was necessary that all data be in an immediately meaningful form* That* is to 

J - . i • ' ' 

say'^ that we did not Have the time to collect and analyze group performance data 
necessary to providing normative and^ comparable meaning to the variables 
assessed. For that reason, to make ^mmedlate decisions possible, we were limited 

to the use of presently normed instruments. 

* . f ■ \ 

Variable Selection 

A major problem in assessing learning disabilities (LD) is that of selecting a 
reference system or systems of learning-related behavior. Such systems are 
comprised of specified constructs, — traits or types of behavior — and specified 
interrelationships among them. Two types of reference systems are widely reflected 
in the LD literature: clinical approaches consisting of numerous, often unspeci- 
fied patterns of responses; and a dymptomatic typological approach consisting of 
relatively well defined components of LD — > aphasia, dyslexia, hyperkinesis, and 
so on. Neither approach seemed adequate for this study — the former being 
largely idiosyncratic to the particular clinician and unrepllcable, and the 
latter casting a net that is agreed upon as too narrow and somewhat siTp^rfJrCial. " / 
Most educators would agree that LD consists of more than aphasia, dyslej^^^^^ and 



hypetklnasls and, furthermore, t;hat It^'ia^the shared characteristics of the - 
eyndroitttts rather than their surface syioptoms that are of interest to a complete 
LD conceptualization. Rather than accepting either the weaknesaes of the 
clinical/judgmerttal or syinptomatic/partial definitions, we have chosen variables * 
according to a mor« basic psychological reference system and have selected 
measures (and records of scores) which are intended to. reflect clinical and 
symptomatic/typological points of view as well. 

Within this approach, we had to settle for an* incomplete soliftion since 
testing time had to.be helti to reasonable limits and^vailabJ^e inftfrmation from 
records was only marginally helpful. ^However, we proposed the alJ^ocation 
of 3^^^ajjall portjten of testing time to .gathering data on the^adequacy of the 
reference systems and measures employed. r v 

In the follow^rtg sections the reader will notice "marker variables.'* 
These short research instruments were intended as checks' on the structure of the. 
diagnostic instruments and were not intended for use in making diagnostic decisions 
Correlations betvTeen marker variables and diagnostic indices are being studied 
for convergence an^l divergence, and factor analyses will be performed to confirm 
or deny the existence of the intended structure in the diagnostic battery. 
This use of scarce testing time seemed appfopriat,e due to the admittedly special 
nature of the LD and JD populations. Though we may be well assured that we ' 
understand the meaning of performance levels on digit spati tasks in the normal 
population, for example, we were far less assured regarding .their meanings in 

special groups. The marker tests enhance our ability to provide geologically 

^'^ • . >■ , 

^ r- 
valid interpretations of performance and to describe the structure of traits in 

the study's potentially unique populations. y 

^ 10 . 



In»trmneat Stlactlon 

Xh« oMiMurM lifttfd in tha following chart roflact several approachee to 

U). The theoretical work of Thurstone, Guilford, Cattell, and Wltkln on the 
etructure of abilities guided our thinkings' The factor analytic studlea of. 
Wltkln and hia colleagues (Wltkln» Dyk, Faterson, Goodenough, & Kai^, 1974) were 
deeme'd eifpeclally relevant, as they provided coamion flpdings with the various 

* ■ ' 

theoretlcial structures, and their use of the WISC provided the desired link with* 
widely used clinical Instrument*. In addition, clinical concep.tlonih of LD were 
reflected In the choic^e of the Behder-Gestalt and other petceptual measures. 

■ " The WISC-R .Is so widely uspd and well known that a detailed description Is 
.not needed here# In addition to theoretical considerations (reflected ±«n the 
additional scoring for the V^ltkln, et al fact|ors,), the availability of WISC-R, 
Scores allows for direct comparisons between results obtained In this 
exploratory study and those obtained In many other studies of LD children. ^ 
Moreover, the significance of discrepancies between any two WISC-R subtest 
scores or between the Verbal* and Performance* Scales is well established in the 
literature, thus falling easily into th'e pattern or discrepancy model used 
here. ' » . . . 

In addition to the conventional WISC-R sdorlng procedures*, we obtained 
scores following Witkin, et^ al (1974) that yielded indices of. the following three 
intellectual factors: Analytic Functioning, Verbal Comprehension, and Atitention/ 

Concentration. Four shqrt instruments were used to mark two' of the factor 

- ■ ■. . • ' \ 

scores^ The Analytic Functioning sdore was marked by results on the Hidden 

■* 

Patterns Test (CF-2) and the Children's Embedded-Figu^res Test (CEFT). 



The Nuwiber Comparison Tfest (P-2) and Identical Picture's Test (P-3> were uaed to 
mtk. the Attention/Concentration factor. Because of the robustness o£ the Vertfal 

Covkprehensibn factor and because many-of the achievement measures are so infused^ 

.. ' ' * 

with this factor, no compelling need was seen to provide additi^onal markers of 
Vetbel Comprehension. . . 

The Bender-Cestalt Is anofcher widely used, well known instrument; that 
needs no description here. It, too, will allow for comparison? between results 
obtained in the current study and the LD literature. It should be noted, ' however^ 
that the Bender is suited more to the perceptual ability develdpment of youbger 
children than* those in the current study, and the results may therefore tfe most 
useful diagnostically as a "low pass" screening device- The scoring method 
chosen was that, developed by Koppitz (1964) with a thresh61d of ^our or more tSrrors 
The Thutd tone Flags and the Swinton-Wepinan Test of Memory for Visual Orientation 
provided markers of the Perceptual Ability factor as assessed by the Bender- 
Gestalt Test. ' 

Both tbe^ Woodcock Reading Mastery Test and Keymath Diagnostic Arithmetrlc 
Test were selected because they provided uniquely precise measurement in a short 
testing time. Each was developed using the Rasch model, such that ^?here is a 
commojn 3cale on which ability in a specific area and the difficulty pf specific 
itemfl can be represented. As in the WISC^R, all" subject S; began with .very single 
itensB and continued until they responded incorrectly to a given number of consecr 
utive Items which have been scaled and ordered a^ccording to difficulty. 

In view of the established association between hyp^rkinesis and tmpulsivity, 
indicators of impulsivity. were assessed through the technique of error-^scoring on 
thre€i Igfistruments: the Coding subtest from the WIS'C-R; the Number Comparison 

V 
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the student's record to serve as construct validation data. 

There are, of course, myriad other Instruments that were considered and 
that might have been selected for this study. Those chosen for the assessment 



literature on LD, 

Diagnostic Decisions and Incidence Estimates 

Judgments of LD In the adjudicated JD samples had to made Immediately following 
Individual assessment so that LD subjects could be assigned to the experimental 
conditions of the ACLD remediation effort, which was to start as soon as possible. 

It was theref ore^ necessary to consider a^ priori models that describe LD 
in terms of levels of performance on the single Variables selected, In terms - 
of relative levels — discrepancies — , or In terms of more complicated patterns 
Involving relative performance or performance levels on numerous selected 
variables. The selection of such an a priori model was neither easy nor rewarding, 
for there were myriad proposed candidates ranging from simple ratios or differences 



between two variables to the detection of complex "within test'* patterns of 
behavior, apparently obscured from those other than the numerous au thor-cllnlclans 
writing about them. The probability of finding a reasonably correct, supportable^ 
or popular a priori definition was either terribly small or Incalculable; but we 
had to provide diagnoses that were made according to specified, repllcable decision 



battery are Integral components of the specific reference system established by 
Wltkln, et^ al (1974) and are represented in a large portion of the clinical 




rules* Further^re, these rules had to be sufficiently uncomplicated so that 



some assurance could be maintained in their proper application under field 
conditions in the hands of trained-, supervised staff. The rules had to actomito- 
^d^te.both records and individual assessment data. \ 

We are all aware that, the term **LD" ia general, ambiguous, variously defined, 
and rarely expressed in terms of a replicable decision rule. Lemer (1971) 
records the following variations: 

1. Neurological dysfunction or brain impairment. 

2. Uneven growth patteims. .:i 

3. Difficulty in academic and learning tasks. 

4. Discrepancy between^achievement and potentiality. 

5. Definition by exclusion. 

Also, Murray, e_tal (1976) have provided another definition as follows^ 

1. The diagnosis should be based on evidence which cannot as 
easily: be interpreted a s a manifestation of mental retardation, 
physical handicap, emotional disturbance, or environmental 
disadvantagement. This does not mean th^t each individual 
indicator must be unambiguous, but that ^he diagnosis should 
be based on triangulated measures which permit a pattern that 
is inconsistent with the alternative explanations. 

2. The diagnosis should be accompanied by evidence that a 
discrepancy exists between achievement and expectation. For 
example, that a child may be demonstrated to occasionally 
reverse letters does not constitute a learning disability if 
the child is reading and writing at the level expected of 
that age and intelligence. 

Finally, the proposed definition of the Bureau of Education for the Handicapped, 

Severe Discrepancy =* C.A. -h Q.17) - 2.5 

has stimulated intensive discussion (Federal Register, 1976). 



-10- 

While all of these definitions include the idea of a discrepancy, none except 
the BEH formula can be applied without resorting to rather extensive clinical or 
psychoeducational judgment^. The BEH formula, however, does not take into Account 
possible additional evidence which, considered with more .moderate discrepancy 
than the formula requires, might indeed identify children with serious learning 
disabilities. For the purposes of this study, we used a^definition which 
includes the principles suggested by Murray, et^ al and extends them through 
specification of replicable decision rules. The definition accepts intact the 
first part of the Murray's two-part, definition and specifies the evidence required 
for possible diagnosis. 

In the following sections specific decision rules for evaluating the evidence 
are described. They were based upon level of performance, in some instances. In 
record review, for example, JD's with academic achievement commensurate with their 
ages were removed from further consideration as potentially learning disabled. 
Also, we recognize certain high ability LD youths might be functioning at reason- 
able levels of academic achievement. However, specifications for fhis study did 
not provide for their inclusion. In those instances where level was of importance, 
measurement error was specifically taken into account. A second type of decision 
related to discrepant performance on two or more variables. Discrepancy-based 
decisions predominated in the treatment of assessment data and took into account 
both the statistical and the substantive significance (educational, psychological) 
of the magnitudes of differences. The statistical treatment was imperfect, 
however, for the iritercorrelations among measures used were unknown. Finally, 
data from perceptual tasks were used in a confirmatory sense because adequate 
normative information on the measures selected was not available; and we were not 
able, therefore, to transform scores to a scale comparable to other measures. 
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Specifically, the Bender-<3estalt was used In conjunction with ability and achieve- 
ment levels or discrepancies to affirm or deny perceptual Involvement In apparent 
difficulties. 

Discussion and specifications In the following "Process" sections provide 
decision rules In precise detail. A preview here might point out that a delin- 
quent who was judged LD was not mentally retarded, nor severely emotionally 
disturbed, nor achieving at an age- and experience-appropriate level. His 
performance on assessment measures showed discrepancies at least as large as the 
difference between the means of age groups two years apart. Observation during 
the assessment provided soine assurance regarding the youth's vision, hearing, 
and freedom from obvious physical handlcapy^ as well as some data on potential 
behavioral correlates of LD and test-tak.lng behavior which assisted In test 
Interpretation. This seemed to us to be a reasonably large infojrmatlon yield, 
given our limits of testing time. 
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' HI: PROCESS 

* 

Records Review * * > r 

After sample selection, -sets of actual records were used for training the , 
records reviewers. The review process Involved, first, the recording ^f relevant' 
data on subsequently described forms and then followed a systemiatlc decision 
process, assigning each record to an appropriate category. The review resulted 
in three categories of decision: those youths who were screened from further 
consideration because of mental retardation or primary emotional disturbance; 
those who were interviewed; and those who were diagnostically assessed. Th^— ^ 
branching of the review process at various decision points is Illustrated by 
Figure I. Decision points are numbered on this figure.* As mgtjpaoned previously, 
two prior decisions applied to the public school sample and one to the adjudicated 
delinquent sample. The decision common to both samples was age appropriateness. 

The records of youths born after March 1, li865 and on or prior to March 1, 1960 

> 

were eliminatea from further review. For the public school sample, youths ^ 
with records of adjudicated 'delinquency were cross checked against court records 
and, if appropriate, included in the delinquent sample. Also, designation of 
visual, hearing or speech probleqis in the records were noted on the basic data 
jiform for possible consideration at a later date. In some cases records were 
Inadequate for the record reviewer decisions. These were categorized for diagnos^ 
tic assessment (DA). 



*The numbers are not in strict ordered sequence because bran(;hing and recycling 
prevent such numerical representation. 

**The study was originally designed for 12 to 15 year olds. The sample was 

expanded to include 16-year*olds and youths who had just turned 17-years-old in 
the record review process. 
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The first decision rule required consideration of a recorded clinical 
diagnosis of learning disability (LD), If such a Judgment was in the record, the 
youth was assigned to the category for DA% If it was absent, the second decision 
rule concerned a diagnosis of mental retardation (MR). We selected the cut-off 
score of 75 on the most -recent psychological examination for this determination, 
with the further provision that if a discrepancy within |the psychological test 
profile of greater than 9 points existed, students with composite scores of 68 or 
higher were considered further. If the test used in the determination of mental 
retardation did not provide part scores, the record was examined for achievement 
profiles whi^h were unusually mixed or other scores such as the Bender-Ges talt , 
which had been interpreted as suggesting perceptual and/or general learning 
problems. When these were found, the record was subjected to further review. 
Otherwise, a Judgment of MR was made and the record was categorized as screened 
out of the study. 

The third decision rule applied to emQtional disturbance (ED). If there was 
no record of ED, the record received further^ sereenitng. If an ED diagnosis was 
present, it was evaluated for severity. There is considerable uncertainty about 
the primacy of emotional disturbance in learning disabled students. This study, 
therefore, included students so diagnosed unless there was evidence to suggest 
the impossibility of securing valid information from the student because of 
profound disturbance. For both JD and non-JD students, if the ED diagnostic 
report contained in the record characterized the student as with3¥awn, unable to 
relate to the examiner, generally untes-table ^r~-p resenting unusually bizarre' or 
severe symptoms, including a record of hospitalization for these symptoms, the 
student was categorized as ^creened out of the study. If the behaviors, on the 
other hand, were more passive^ such as shoplifting, , taking small amounts of money 
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from parents, lying or.tiruancy, the record was reviewed further. The diagnostic 
supervisor made a clinical judgment on each case which might be ruled out because 
of ED. ^ 

For those records subjected to further review, the fourth decision riile 
iponsidered achievement scores. Where these were available, the levels of achieve- 
ment scores were considered. Records of achievement scores which were at or above 
a point two standard errors of measurement below the mid-point score for children 
of Similar school experience were placed in the interview category. All others 
were reviewed further. 

Decision rule five concerned profile discrepancies, both between ability and 
achievement scores and within achievement sco re profiles. While i t wou Id have 
been desirable to determine the significance of a profile difference through the 
use of a standard error of a difference formula, tfcie variety of tests occurring in 
th'B records and the lack of certain necessary statistics for these tests rendered 
such formulae impractical. In particul^r^ the intercorrelations between all 
possible pairs of ability and achievements tests were unavailable. The possible 
standard error bands therefore had to rely on formulae which did not take into 
account intertest correlation, but required only the variances and reliabilities. 
The formula used is available In many standard texts. Its application required 
conversion to a common scale, under the assumption of a normal distribution. 

The particular version was adapted from Stanley's (1971) formula VlT^ The 
signficance of differences was established by this method, but the method 
did not determine the decision. Rather, the width 6f the band was set by what we 
call 'Miagnostic significance." In the case of the ability /achievement discrepancy 
this value was determined by a score spread which represented test performance 
equivalent to two years' difference. Two years was selected because the perfor- 



mance of urban popUflatlons , heavily represented among adjudicated delinquents, 
tends to be, in general, about one year below the norm on the average. This 
difference can be reasonably attributed to environmental factory, and conse- 
^souentiy niust)be allqwed f or . in revlewi-ng discrepancies. When a significance 
tdst was applied to these differences, using equivalent scores and reliabilities f 
f^^m, for example, the Stanford Achievement Test-Advanced Battery, we found that 
the differences were reasonably unlikely to- be artifacts of th^^fsting. 

To repeat, decision rule five required that if the achievement score differed 
by the equivalent of two years from the ability score on one or more tests, the 
record was placed in the category for diagnostic assessment. A mixed achievement 
profile with two-year discrepancies also resulted in classification for diagnostic 
assessment. If, on the other hand, no such difference was discovered, the record 
was reviewed for evidence of hyperactivity,, unusually illegible handwriting, 
perceptual or motor performance tests which indicated malfunction and, if grades 
were available, for erratic grade profiles (decision rule 6). Also significant 
for this rule were abrupt changes in profile character over time. That is, 
achievement mi have been reasonably consistent through the early grades, but 
become erratic in the later grades. if any of these were present, the student's 
record was placed in the category for diagnostic assessment. Otherwise the 
record was placed in the category for interviewing. 

For those records which showed an achievement score profile but no ability 
score, decision rule four (level of achievement) was applied. Records falling 
within the achievement limits were considered for grade profile if grades were 
a^^ailable (djacision rule 7). A profile which was above average was categorized 

f ' 

for interview. If the profile was average or below, the grade profile character 
was considered (decision rule 8). A mixed profile was defined as one which 



contained grades In reading, language, math, and. science which, for two separate 
years, showed deviations of at least two letter grade intervals. For example, 
a letter grade profile CCBD was considered mixed, whereaW-C^C was not. Other 
marking systems were converted to a five-interval scale, with the failing point 
being considered the top of the lowest interval, and used in an anAogous manner. 
If the institution assigning the grades reported its own interval scale in a 
usable form, these intervals were applied. When a mixed profile? was present for 
two years, the student was assigned to the category for diagnostic assessment. 
For those records which did not show a mixed profile, decision rule six was 
applied. In records for which grades were not available, rule six was applied in 
terms of the indicators other than grades. 

^ In those cases where ability scores but not achievement scores were avail- 
able, the availability of grades was considered. If they were available, the 
categorizing decision was made in terms of decision rules seven, eight, and 
six, as applicable in the rule-determined sequence. Where grades were not 
available, decision rule six was used to assign the category. The same decision 
process applied when neither achievement nor ability scores were available. It 
should be noted that in no instance was a student screened out of the diagnostic 
assessment category on the basis of a group ability measure alone. It should 
also be noted that decision rule six was applied as the final test when any other 
data suggesting learning disability were lacking. Whenever insufficient data 
were present to make a decision on these rules, the student was diagnos tically 
assessed. 

The entire records review process was under the direction of the diag- 
nostic supervisor, a certified diagnostician. This person paid particular 
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attention to the decisions about emotional disturbance, personally reviewing . ' 
clinically those cases where the basis of the recorded diagnosis was not obvious. 



The Interview process iSo described next. 
Interviews 



Interviews were conducted following the records review process for all students 
except those who were not previously screened out because of evidence of mental 
retardation or emotional disturbance. Students were Interviewed whose records 

s 

indicated (1) higt\ achievement scores, (2) consistent high grades, or (3) 
non-discrepant academic profiles with no other LD indicators. Students who 
were diagnostlcally^^ass^sed were interviewed immediately following testltig. 
Interviews were recorded on special forms incorporating contenf provided by 

Greighton Institute. The general purpose of the interview was to examine whether 

'\ 

students with learning problems get into more trouble than students without 
learning problems, as well as to collect pertinent background infonoat ion about 
the participants. 

The first phase of the interviewing process involved training the inter- 
viewers. The preliminary part of this training, which was also prerequisite 
to ensuring successful interview experiences, involved learning how to establish 
rapport with the students. The diagnostic supervisor trained interviewers in 
techniques of establishing rapport, which InMuded such topics as (1) general 
characteristics of the adolescent (social, emotional, physical), (2) potential 
psychological problems inherent in the adolescent /adult interaction, and (3) 
desirable Interviewer characteristics ^ (e. g. , voice intonation, language usage 
level, eye contact, authoritative attitude level). Part of the interviewers' 
training involved participation in hypothetical interview situations -from the 
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posltloti^of either the Interviewer 6r the student, and responding verbally about 

how one would de'al with a stated problem which n^ght be encountered during an 

Intejrvlew* In paiftlcular, this training stressed the Importance of ensuring the 

student that confidentiality would be protected. 

Each interview ^ s pent approximately 20 minutes with each studetit in the ( 

actual i|tt.ntervlew process. This 2D-mlnute block of tln^ for Interviewing was 

^Ivl^d as follows using interview content provided by ^^relghton Institute: 

Time (minutes) 

Rapport building 5 r 

Other designated interview ^ 
' schedule components 15 

2^ 

At the completion of the interview/* the Interviewer reinforced the positive 
Intent of the situation by thanking the student for his assistance, attention, 
and cooperation. Completed interview forms were collected by the diagnostic 
supervls<f>^ and the data were later forwarded with the diagnostic data to Crelghton 
Institute. \ 

Diagnostic Assessment and Decision Process 

The role of the diagnostic assessor, although seemingly limited because of 

its very specific performance guidelines, was vital to the collection, recording 



and transmission of accurate data. Performing this role successfully subse- 
quently required such things as: 

:K Working familiarity with each test in the battery. 

2. Adherence to standardization requirements, with sensltlvi^ty to the 
population being tested (e.g., pronunciation differences). 

3. Accurate timing, where required. 

4. Consistency in verbal feedback to student responses. 
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Maintaining eye contact with the student, 

6, Prep,aring test materials prior to meeting the stOdent* 

Specific instructions included the following admonitions: 

• - ' - .-, . ^ 

Remember that adolescents in general, and particularly institution- 
alized juvenile delinquents and learning disabled youth, tnay tend to give up 
' easily and thus may need continuous encouragement . ^ You should also be aware; 
thit certain questions on the WISC-R may be threatening in conteirit or 
presentation form to adjudicated delinquents. Specifically, we suggest ^ ; 

asking "what do people say a 'thief Is?" in the Vocabulary subtest ^^and'firf 
the Comprehension subtest, we a^so suggest introducing questions 4 and 9 
with "what are some reasons that people ^ay^i ". It would be well to 
remember, too, to keep response sheets^and manuals with visible answers from 
the open view of the student during testing. Other than for these minor 
modifications, the test administration and scoring directions should be 
f oJ^lowed verbatim. 

Also included in your role as a diagnostic assessor Is the administra- 
tion of a short interview form. This instrument, the content ofjwhich has 
been designated by the Creighton Institute, has been modijEled In an attempt 
to reflect contemporary wording, values etc. , but cannot be changed in content* 
Be sure to fill In the st^dent's Identification Code number at the top of 
the form before beginning* Responses will be marked directly on the form by 
filling in the blank, circling the number of correct responses, or placing a 
check or X under or beside the response given. The Introductory remarks 
should be read verbatim but can be expanded slightly to facilitate coopera- 
tion and trust, but exact item wording should be presented verbatim, if ^ 
possible. 

As with the test instruments, it is to your advantage to be familiar 
Hith the items' on the interview guide prior jto administration so as to be 
able to maintain a relaxed but sincere atmosphere. You should remember to 
thank the student for his cooperation, assistance and patience at the 
completion of the assiessment/intervlew session. 

The diagnostic assessment sessions began with introductions and rapport 
building activities similar to those utilized for the interviews as described in 
part A of this section. The tests to be administered Included the WISC-R, 
following standard procedures, followed by* ap achievement test in reading and in 
math; the Bender-Gestalt , Rosner's Audit bry^ Analysis Test> and six short data 
collection devices which were' selected' to provide corroborative and analytical 
information in support of three major components of the assesamenA The Interview 
questionnaire completed the data collection. 
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The teats were administered in the order presented below to vary the nature 
of the type of tasks Introduced to the student and, hopefully, to increase his 
'Mfillingness to Cooperate over the three hour testing and interviewing session* 
Specifically, the order in which the battery was presented was: 

1, WISC-R 

2, Bender- Visual Motor Gestalt Test 

3, Woodcock Reading Mastery Tet 
Bosner's Auditory Analysis- 

5, Hidden Figures Test 

6, Key. May Diagnostic Arithmetic Test. 

7, Children's Embedded Figures Test (Part 2) 

8, Number Comparison Test 

9, Hidden Patterns / 

10, Swinton-^Wepman Visual Orientation Test 

11. Thurstone Flags V- 
After the session, the student then returned to his school or home, . 

When the student had left, the diagnostic assessor entered his or her 
observations of behavioral Indicators on the records form. The behaviors ot 
interest are shown in^Figure 11. They included four observations specific to the 
WfXSC*-R, The diagnostic assessor then scored each of the test answer sheets and 
entered the results on the basic data form. ^ . , 

Finally, using the decision rules discussed in Section II, the diagnostic 
assessor recorded a coded judgment of LD/not LD on the Basic Data Form, 
these 'decision rules required the conversion ,of the scores of each test to a 
common scale. Under the assumption of a normal distribution, and given knowledge 
i^?if the mean and standard deviation of each test, a direct conversion to a 
^Standard scale was made; In the interest^ of convenience, the "T" sc^le, 
with a mean\ of 50 and a standard deviation of 10, was selected. Conversion 
cables were provided for direct Look up and entry on a profile form. 

The first consideration in the LD/not LD Judgment was a review of the 
profiles for discrepancies ac Least equivalent to the difference in group means 
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of groupa two years apart • These differencas may have occurred within the score 
patterns of the WISC-R, including .the Witkin factor scores, between the WISC-R 
scores awd any or all of the achievement scores, or between the achievement 
scores. When discrepancies of the indicated, magnitude existed among all, three . 
sources of score data, the decision was clearly LD, When two sources showed 
dlscrepamcies , the same decision applied if there was any supporting evidence^ 
from .Che Bender or the observations. When only one pair of scores showed discre- 
pancy, supportive evidence from the Bender and two or more pronounced character- 
istics from the behavioral observations were required for LD diagnosis. Cases ^ 
which showed no significant discrepancies, as defined, but demonstrated two years 
below level achievement and included observations of difficulty in following oral 
directions, motor difficulty^ paper rotation, productive language problems, 
distractability, and at least one of the WXSC observations were also judged LD If 
the full scale WISC'-R score was at least 75. Cases which did not meet any of 
these criteria were judged non-LD. In order to maintain the greatest degree of 
Independence in subsequent judgments of the protocol, each diagnostic assessor 
was provided with a sealed envelope containing a positive and negative LD code 
number for entry on the form* During training careful emphasis was placed upon 
the Importance of attaining independent judgments. The code assignments were 
retained in Princeton, and a periodic check of their consistent application was 
conducted by persons not otherwise involved in the diagnostic assessment* 

After the Basic Data Form was completed, it was given to the Diagnostic 
Supervisor for secure storage until the first data review was conducted. This 

review occurred In general on every fifth day of assessment. Each diagnostic 

J* 

assessor was given a set of data forms to review for students assessed by otliers* 
The decision rules were applied and a judgment of LD/not LD was made by the 
reviewing aaeressor. ^ 
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It slibtfld be noted at this point that lu some cases the decision rules did 
not provide «n unambiguous conclusion. In these cases the diagnostic supervisors 
were instructed to resolve the situation with a clinical Judgment based on 
all the relevant data which were recorded on the form. 

When thlaljfinal Judgment was recorded, the basic data form was sent to • 
ETS Princeton for transcribing onto tape and for random selection' Into the 
remediation and Complarison groups of those adjudicated youth who were Judged 
LD. The recor;5ls of the remediation candidates were then reviewed by the remedia- 
tion diagnosticians, and copies were transmitted to the ACLD remediation teams, 
accompanied by the remediation recommendations. 

, Quality control checks were spaced such that each dlagnostlc^assessor ^ 
received a check on the average of ev^ry eighth day. The checks were conducted 
by the diagnostic supervisor, who spent the. entire assessment session with 
the diagnostic assessor. To reduce the Impact df two adults working with one 
student, the diagnostic supervisor greeted the student, escorted him to the 
assessment room, and introduced the diagnostic assessor. The supervisor then told 
the student that both adults would work with him during the session, and occa- 
sionally contributed supportive comments. Sufficient time was provided in the 
schedule for occasional group discussion of the assessment process. 
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IV. ADMINISTRATIVE AdRANGEMENTS 
Major Tasks and Staffing 

Following sample 8election» the implementation of this study required completion 
of several main tasks. They were: review of school and court records of students 
in the respective samples for evidence of possible LD; interviews to collect 
background and experience data; diagnostic assessments to provide the information 
to cdnfirm or disconf ixrm the possible LD hypotheses indicated by the records 
review; review and decision based on these data; analysis ^ records and assess- 
ment data to provide remediation suggestions to the ACLD remediation team; and 
assessment of the remediatioi^ and comparison samples for post-treatment data* 

These tasks^ were accomplished by teams' of professionals under the direction of 

I 

diagnostic supervisors, as will be the posttesting after the completion of the 
remediatlop program. 

The diagnostic supervisors at all three sites were practicing certified 
psychologists. The supervisor of the Baltimore site also served the function of 
Associate Project Director. In addition to the required credentials and. active 
engagement criteria, the supervisors also met the qualification of being exper- 
ienced in working with adjudicated delinquents. 

The diagnosticians had equivalent qualifications to those of the supervisors. 

In addition, experience in, educating LD children was expected of this group. The 

=* ■ 

primary role of the diagnosticians was to review the teat protocols and the BDF's 
for purposes of providing recommendations to the ACLD remediation team. The 
reviewexs, interviewers and diagnostic assessors were people with degrees in 
psychology, education or similar fields and experience in working with educa- 
tional records and with adolescents. Figure II describes the qualifications and 
responsibilities, of every member of the staff. 
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.Throughout moat of the pre-^test phase, we found that a team of five record 
reviewers and eight diagnostic assessors was kept busy at all times. In most 
cases the record reviewers also served as interviewers although an additional one 
or two 8taf£ members served in the interview function* 

Training 

— " — t 
Training sessions were most significant as procedures for assuring uniform 

data collection. The records reviewers /interviewers and diagnostic assessors 
were trained on-site by the site diagnostic supervisors and by ETS staff. 

Training began with a thorough review of materials and procedures. The 
records reviewers /interviewers were trained first, since their work began ^ 
almost iitamediately. The record reviewers administered the interview form to one 
another and performed dual record reviews as part of this procedure. Following 
this« the diagnostic assessors (OA) were trained and a progiram of trial assess- 
ments among the DAs was conducted. Each OA administered and was tested with 
the diagnostic assesstbent package. Finally, a period of testing by a pair of 
diagnostic assessors with selected cases was conducted at the outset of the 
diagnostic period. (This served as an initial quality control check; quality 
control is further discussed in the next section.) A final crucial element in 
the training, interspersed throughout its course, involved discussing suggestions, 
anecdotes, and procedures which w^re useful in working with adolescents. These 
were provided by the experience of the diagnostic supervisors and bur consultants. 

Quality Control j 

Any data collection which is as extensive and complex as that designed for 
this project requires careful attention to assure the quality of the data* 
Several procedures were designed to accomplish this purpose. First, 3 to 6% 
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of all diagnoses (spaced across the entire period of assessment) were quality 
controlled by the diagnostic supervisor; this was accomplished by the actual 
presence of the supexrvisor during the required number of diagnostic assessment* 

Second, a one-half -hour protocol check of each diagnostic assessment was 
made in a cooperative effort between the diagnostic supervisor and the diag**' 
nostic assessors* (The assessors did not check cases which they themselves 
originally assessed.) This protocol check consisted of a review of the entire 
data form, including school or court data transcribed thereon, assessment data, 
the application of the decision rules » the assessor's comments and the recorded 
observations made by the diagnostic assessor who collected the data. This review 
lead to an independent Judgment of LD or not LD by the reviewer. The first 
review was ''blind" because the Judgment of the data collector was entered in 
code. The diagnostic supervisors reviewed the two DA decisions to identify and 
resolve any difference in Judgments. 

Third, a second protocol check of the LD/JD diagnosed cases at each 
site was made by the diagnosticians. This check was part of the review process 
which produced remediation recommendations. 

Finally, periodic- site visits by a two- or three-person monitoring team 
(members of the advisory committee) were conducted during the diagnostic testing 
period for purposes of reducing variation in diagnostic points of view. The 
monitoring team obsexrved actual assessments taking place at each site. 

The team members were J. Richard Harsh, a clinical psychologist who has both 
worked in California's Juvenile Halls and has conducted extensive research wltih 
another team member. Professor Nadine Lambert » of UCLA Berkeley. 

There were also cross checks of samples of deidentified protocols across' 
sites » followed by an analysis of these cases to achieve greater commonality and 
to check for diagnostic drift. This entire process can be classified as an 
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additional quall'^y control check. It had been suggested that external evaluators 
Conduct this filial review. However, Its prime purpose was to facilitate cross- 
«lte uniformity pJ^ovldl^g commoii' sets of data for discussion. The direct 
Resolution of differences of opinion among the dljagnostlc supervisors was a part 
Of this process* ' 

^onns 

I. Basic Data 

All data gathered throughout the record screening and diagnostic assessment 
Processes were recorded on an Individual f6nn, which, with the attendant answer 
Sheet(s), provided a complete diagnostic package. separate form was used by 
the interviewer* It was later merged with the records data.) 

Part I of the form^was used to enter information found, in the subject's 
School and court records. ^Yits information pr6vlded the tool for decision 
iJiaking in response to screening criteria as they were outlined ^he decision 
tree (see III. PJ*ocess Recprds Review). These criteria Included prior 

diagnoses of physical handicap , emotional disturbance^ mental retardation. 
Or LD, as well as ability achievement data, recorded grades, profile character 
^nd conanents reg^^dlng LD syuiptoms. Also included in Part I were spaces for 
I'ecording speech* vision, and hearing problems 

Part II was used to record the student's performance on the testing Instru- 
Jtients. The test instruments were hand-scored by the DA to produce this record. 
The DA entered a coded judguient of his/her opinion as to whether the b.oy had a 
learning disability. 

Part III pt the form was used by the diagnostic assessor for written 
Observations of ^^ch student's behavior throughout the assessment. Provisions 
Were made for coonnents by the diagnostic assessor or the supervisor. 
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Part IV of the document contained the overall Judgment of a second diag- 
nostic assessor and/or the diagnostic supervisor. This summary was based upon a 

^half hour protocol cheese and resulted in either a "non-LD" or "LD" decision. . 
Also included in Part IV was an indication of current detention status, i.e., 

'none, probation, parole, or institutionalized. t 

2. Profile and Summary Sheets 1 

a. To fiacilitate comparison of information from several sources, signif- 
icant data were entered on a profile sheet. The major elements in this profile 
were the five scores from the WISC-R (Verbal^ Performance, Analytic Functioning, 
Verbal Comprehension and Attention/Concentration) and the achievement scores in 

r 

reading and i!)athematics. ■ ' 

b. A summary sheet was used by the diagnostician for all remediation cases 
who were assessed LD. (Note the exclusion of non-JDs in this final diapiostic 
process, since none of these cases were included in the remediation sample.) 
This summary sheet provided handwritten data recorded during an additional 
protocol check (one hour) made for each youth who was randomly selected for the \ 
remediation sample. Its major purpose was to provide recommendations for remedia 
tion for the student^ and it was included in his diagnostic package as part of 
the Basic Data Form. ' . 

•r 

* 
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V. APPLICATION 

^- The pro^vgdures outlined above w^re applied at three sites; Baltimore, 

Indlaii^p^^3 and Phoenix- The public schools, juvenile centers and Juvenile 
, Instittt^i^ cooperated In eacl> case. At; the date of this writing the study Is 
still utidervay. 

, , S6nie Interesting ^^scellaneous sidelights aight be useful to those who 
antlclp^^^ Similar studies: , _ 

~ experienced records reviewer can averag® about eight records per day, 
^^Cords are stored In a variety of locations, are hot necessarily 
^^^tact. Where th^ are wen organized, as nany twelve records per day 

be reviewed u^lng thes^ procedures. Review Includes recording pertinent; 
^ata on the Basic I^ata Form and making the <i|clsion to asses or not to 

^s^ess, '"^1 

I'- 

~ schedules of schools are such -that ;the coflq)letlon of two 4lagnoatlc 

' ji ■ ■ ■ 

Assessments per d^V Is frequently Impossible, average of one and one 
^^If per day Is a reasonable expectation. 
~ Th^ only effective way to arrange assessments and Interviews with parolees 
And prol^atloners is througi^ dlrQx:t perspn^l contact. Ample personnel time " 
for guch an approach should be provided. 
~ There are remarkably few outright refusals to participate on the part of the 
\. ^-tudejnts once arr^^^genients made. Genuineness, warmth and'patlence on the 
part of the dlagn^^'S tic assessors and Interviewers prolixly contribute most 

the successful i^lenientatlon pf the proccdur^g^ . 
"^h^ collection of these kin^s of data In a field operatldii Is a highly 
person-ij^^enalve activity, and vill Inevitably ^® a costly process. It Is, 
^^owever, the only avall^t)le method which Is applicable for research on the 
kinds Of questions addJ'^ssed by ^hls study. 
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Figure 1 ^ 
Selected Instruments 



Intellectual Functioning 

WXSC-R (Conventional scoring) 
t 

Varbal Scale 

Information 

Similarities 

Arithmetic 

Vocabulary 

Comp r eh ens ion 

Digit Span 
Verbal 10 

Perfoo^ance Scale 
Picture Completion 
Picture Arrangement 
Block Design 
(Jbject ^ssejBbly 
Coding . 

Performance IQ 

Juli Scale IQ 

WISC-R (Additional Scoring) 

Analytic Fu'nctibning 
Block Design 
Picture Completion 
Object Assembly 

> « .Verbal Comprehension 
Vocabulary 
Information 
Conqplrehens ion 
Similarities 

At tent ion /C oncen t rat ion 
Di^it Span 
Arithmetic 
Coding . 

Factor Markers v 
Hidden Patterns CF-2 
Children's Embedded Figures 

Test (CEFT)- 
Ni^er Comparison P-2 
Identical Pictures P-S 
Thursitone TFlags 
Test of Memory for' ; 

Visua!l Orientation 



Items 



Time 
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30 
17 
18 
32 
17 
14 



26 
12- 
II 
4 
45 



11 
26 
4 



32 
30 
17 
17 



14 
18 
45 



400 

14 
96 
96 
21 

20 



A5-60' eat 



3' 

10' 
3' 
3' 

• 5' 

5' 
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Perceptual Ability 
B ende r --Ges t a 1 1 . 
Rosner's Auditory Analysis Test 

Achievement 

Reading 

Woodcock Reading Mastery 

Letter Identification 
♦ Word Identification 
Word Attack j 
Word Comprehension 
Passage Comprehension 
Total Reading 

Arithmetic 

Key Math Diagnostic Arith- 
metic 
Content ; 
Numeration, 
Fractions 

Geometry & Symbols 

Ot>eratlons ; 

Addlton 

Subtraction 

Multiplication 

Division 

Mental Computation 
Numerical Reasoning 
Applications ; 
Word Problems 
Missing Elements 
Money 
. Measurement 
Time 

Additional Observations 



Items 



9 
20 



45 
150 
50 
70 
85 



24 
11 
20 

15 
14 
11 
10 
10 
12 

14 
7 
15 
27 
19 



Time 



15' 
10' 



20-30' est 



30' est 



Vision 

Hearing 

Impulslvlty 



Total estimated time for entire battery: 141 - 166 minutes* 



The standard dlreiptlons for the. marker tests have been modified as necessary « 
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Figure It 
Diagnostic Staff 



Title* 



Diagnostic Supervisor 



Responsibilities 

Supervises, selects, 

trains site team 
Performs quality 

control 
Responsible for maintain-* 

ing security of data 

at site 
Contact person for s'chools 

and institutions 



Qualif icatioas ^ 

Practicing certified psych- 
ologist 

Experience la working with 
adjudicated delicMiueats 

Experience in teaching or 
training of testers 



Records Reviewer 



Interviewer 



Diagnostic Assessor 



Reviews school and court 

records 
Enters data in forms / 
Applies decisions rul^s 

Administers the question- 
naire 



Administers the assessment 

package 
Provides reviews of basic 

data 

Administers the posttest 



Degree in psychology, e4ucar- 
tion, or similar fields and 
expeifience in working with 
adolescents and with educa- 
tional records 



Diagnostician 



Reviews protocols for 

remedial recommendations 
to the ACLD 



Practicing certified psych- 

' oiogist 

Experience- in working with 
adjudicated delinquents and 
with learning disabled 
children 



^or study purposes. ^ . 

^It is understood that the Diagnostic Supervisor at the Baltimore site was also 
the Associate Project Director. 



36 



Records Review 



Fi|ur«;mr 



1' NLD 
2. HS^ 



m 



Screen 3, ed 
out' 

Sqre'en 
out 



NED 



Achievement 
Ability 



Yis 
Yea 



8 



Id 



4. Within 



0ut8ide(HiiicM within 



OuWl^eOUfAch.) 
jivl 



/ 




5. Even Hixed 



In|er- H 

vi'^w 
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Figure IV 



Occasionally 
Pronounced Observed Not 

Characterljstlc . Characteristic Observed^ 



Difficulty following oral directions " 

Low Frust^rttlon, ^-g*^ early onset of 
. fldgttl^ngy Inattentlveness 

Guarded response style (may be 
withdrawal, hostile response, 
evasive response) ^ 

Repeated verbalization of Inability 

to learn ' 

Gross motor .difficulty y e.g.» unusual 

awkwardness 

Fine motor difficulty, e.g., difficulty with 

handling pencil or similar tasks 

v.' 

Manifestation of vision problems, 

e«g. , squinting, holding books very 

close, rotation of paper - ^ 

Manifestation of hearing problems, 

e.g., f^itVorlng one ear, focusing on 

speakers lips ^ 

Continuous rocking, tapping, drumming 

Difficulty in Oral Expression, e.g., 

disjunctive sentences, incoqs^lstent ^ * 

. Z^ram^ lottgr^latency for 

c6ttinon~ words 

DlstractablUty^ ; 

Additional Commehts 



' Wise Observations 
Block designs - perseveratlng In patterns 
Inability to perform on sequence test 
Inability to complete any math problem 
inability to complete puzzles 
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